Selection of encoded data, setting of encoded data, creation of recoded data, and recoding method and device

ABSTRACT

In macroblocks to which a prediction mode determination step, which determines whether or not a prediction mode applied to the macroblocks is a predetermined prediction mode, and an encoded data selection step, which selects a predetermined type of encoded data corresponding to the pixels at predetermined positions in the macroblocks to which the predetermined prediction mode is applied, are applied, there is provided a bit distribution step for more preferentially distributing bits to a predetermined type of encoded data selected at the encoded data selection step than a predetermined type of encoded data that is not selected at the encoded data selection step in the macroblocks to which the predetermined prediction mode is applied. With these steps, it is possible to provide an encoded data selection method which can limit the encoded data per one macroblock transmitted from a decoding device to a recoding device to less than a predetermined value while suppressing the lack ratio of the amount of information of the encoded data as much as possible.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an encoded data selection method and anencoded data setting method corresponding to the above method as well asto devices of these methods. Further, the present invention relates to arecoded data creation method including an encoded data selection methodand a device corresponding to the method. Further, the present inventionrelates to a recoding method including an encoded data setting methodand a device corresponding to the method. For example, the encoded dataselection method is a method of selecting a part of encoded data outputfrom a decoding device.

2. Description of the Related Art

In an tandem-connected MPEG-2 recoding device that uses a decoded imageof MPEG-2 bit stream output from an MPEG-2 decoding device as an input,there is a technology for transmitting encoded data in a bit streamafter it is multiplexed to decoded image (refer to “SMPTE 319M-2000”(authenticated by SMPTE on Jan. 20, 2000)).

In the MPEG-2 recoding device that supports SMPTE 319M-2000 Standard,encoded data multiplexed to decoded data is separated therefrom andrecoding is carried out making use of the separated encoded data,thereby deterioration of image quality caused by recoding can besuppressed as much as possible. Encoded data referred to here is dataobtained by returning code word, which is subjected to variable lengthcoding in a bit stream, to an original numerical value.

In SMPTE 319M-2000 Standard, since it is standardized to superimpose theencoded data to LSB of chrominance of input decoded image,encoded dataof 256 bits can be transmitted by an image of one macroblock.

Referring to Table 2 of “SMPTE 327M-2000” (authenticated by SMPTE onJan. 20, 2000) that is MPEG-2 encoded data standard, since encoded dataper one macroblock of MPEG-2 is 113 bits, it can be sufficientlytransmitted in a band of 256 bits per one macroblock.

Recently, attention is paid to H.264 (ISO/IEC 14496-10). There is a casein which it is required to transform H.264 bit stream encoded by acertain bit rate to H.264 bit stream having a desired bit rate or toMPEG-2 bit stream having a desired bit rate. To cope with this case, itis necessary to connect an H.264 decoding device to an H.264 recodingdevice or an MPEG-2 recoding device in tandem and to transmit decodedimage of H.264 bit stream therebetween. Further, to suppressdeterioration of image quality as much as possible, it is necessary totransmit encoded data after it is multiplexed to decoded image of anH.264 bit stream likewise SMPTE 319-2000.

Incidentally, the encoded data of H.264 per one macroblock is differentdepending on a coding mode of the macroblock.

The coding mode is classified to an intra-mode of in picture coding andan inter-mode of inter picture coding.

Further, the macroblock has a size of 16×16, and as shown FIGS. 1A to1D, the macroblock is classified to four types in the inter-mode. Asshown in FIG. 1A, in a first macroblock type, one macroblock is composedof one submacroblock of 16×16 pixels. As shown in FIG. 1B, in a secondmacroblock type, one macroblock is composed of two submacroblocks of16×8 pixels. As shown in FIG. 1C, in a third macroblock type, onemacroblock is composed of two submacroblocks of 8×16 pixels. As shown inFIG. 1D, in a fourth macroblock type, one macroblock is composed of foursubmacroblocks with 8×8 pixels.

Further, the respective submacroblocks in the four macro types areclassified to four kinds of types. As shown in FIG. 2A, in a firstsubmacroblock type, one submacroblock is composed of one secondarysubmacroblock with 8×8 pixels. As shown in FIG. 2B, in a secondsubmacroblock type, one submacroblock is composed of two secondarysubmacroblocks of 8×4 pixels. As shown in FIG. 2C, in a thirdsubmacroblock type, one submacroblock is composed of two secondarysubmacroblocks of 4×8 pixels. As shown in FIG. 2D, in a fourthsubmacroblock type, one submacroblock is composed of four secondarysubmacroblocks of 4×4 pixels.

As shown in FIG. 1A, in the first macroblock type, one submacroblock of16×16 pixels included in one macroblock is subjected to forwardprediction (F), backward prediction (B), or bidirectional prediction(BI). Further, one submacroblock has one motion vector in one direction.Thus, one submacroblock has one or two motion vectors.

As shown in FIG. 1B, in the second macroblock type, the twosubmacroblocks of 16×8 pixels included in one macroblock is subjected toforward prediction, backward prediction, or bidirectional prediction,respectively. Further, one submacroblock has one motion vector in onedirection. Thus, one submacroblock has two, three, or four motionvectors.

As shown in FIG. 1C, in the third macroblock type, the twosubmacroblocks of 8×16 pixels included in one macroblock are subjectedto forward prediction (F), backward prediction (B), or bidirectionalprediction (BI), respectively. Further, one submacroblock has one motionvector in one direction. Thus, one submacroblock has two, three, or fourmotion vectors.

As shown in FIG. 1D, in the fourth macroblock type, the foursubmacroblocks of 8×8 pixels included in one macroblock are subjected toforward prediction (F), backward prediction (B), or bidirectionalprediction (BI), respectively. Further, although the secondarysubmacroblocks included in each submacroblock share a motion predictingdirection, the motions of them are independently predicted. Further,each of the secondary submacroblocks included in one submacroblock hasone motion vector in one direction. Thus, the number of the secondarysubmacroblocks included in one macroblock is 16 at the maximum, and whenall of them are subjected to bidirectional prediction, one macroblockhas 32 motion vectors.

Note that the submacroblock or the secondary submacroblock acting as aunit of prediction of motion is called a motion compensation block.

The number of bits of encoded data per one macroblock is as shown inFIG. 3. FIG. 3 will be briefly explained from a left side as follows.

-   (1) Intra-mode: 87 bits;-   (2) Inter-mode, first macroblock type, forward prediction or    backward prediction: 52 bits;-   (3) Inter-mode, first macroblock type, bidirectional prediction: 83    bits-   (4) Inter-mode, second or third macroblock type, two submacroblocks    are predicted in forward direction or backward direction: 83 bits;-   (5) Inter-mode, second or third macroblock type, one submacroblock    is predicted in forward direction or backward direction, the other    one submacroblock is predicted in bidirection: 114 bits;-   (6) Inter-mode, second or third macroblock type, two submacroblocks    are predicted in bidirection: 145 bits;-   (7) Inter-mode, first to fourth macroblock types, all the motion    compensation blocks are predicted in forward direction in P picture:    473 bits at the maximum: 473 bits;-   (8) Inter-mode, fourth macroblock type, four submacroblocks are    predicted in forward direction or backward direction: 473 bits;-   (9) Inter-mode, fourth macroblock type, three submacroblocks are    predicted in forward direction or backward direction, one    submacroblock is predicted in bidirection: 582 bits;-   (10) Inter-mode, fourth macroblock type, two submacroblocks are    predicted in forward direction or backward direction, two    submacroblocks are predicted in bidirection: 691 bits;-   (11) Inter-mode, fourth macroblock type, one submacroblock is    predicted in forward direction or backward direction, three    submacroblocks are predicted in bidirection: 800bits; and-   (12) Inter-mode, fourth macroblock type, four submacroblocks are    predicted in bidirection: 909 bits.

Accordingly, since the encoded data of H.264 per one macroblock is 909bits at the maximum, it cannot be transmitted in a band of 256 bits perone macroblock prescribed by SMPTE 319-2000 standard.

Accordingly, to realize recoding while suppressing deterioration ofimage quality caused by the recoding as much as possible, any one of 1)a method of expanding a transmission band and 2) a method of effectivelydisposing data in a conventional transmission band.

Although the method of the item 1) is easy, when data is furthermultiplexed, it is contemplated that a disadvantage such as formation ofa special visible pattern in an image, and the like. Further, it is alsocontemplated that compatibility with an MPEG-2 recoding device makinguse of a present system may be greatly broken.

As a result, the method of the item 2) cannot help being employed. Whenthe method of the item 2) is employed, there is a possibility that anobject of “suppressing deterioration of image quality as much aspossible” cannot be achieved when data is disposed by an easy-goingmanner (that is, “easy-going data reduction”).

Accordingly, an object of the present invention is to provide an encodeddata selection method and an encoded data selection device which canrestrict encoded data per one macroblock transmitted from a decodingdevice to a recoding device to a predetermined value (for example, 256bits) or less while suppressing the lack ratio of the amount ofinformation of the encoded data.

Further, an object of the present invention is to provide an encodeddata setting method and an encoded data setting method which cansuppress deterioration of quality of a recoded image which is recoded bya recoding device making use of encoded data selected by the encodeddata selection method and the encoded data selection device and shown bya bit stream.

SUMMARY OF THE INVENTION

According to a first aspect of the present invention, there is providedan encoded data output method of outputting encoded data of respectivemacroblocks corresponding to an input bit stream in a predeterminednumber of bits or less, the method including a prediction modedetermination step for determining whether or not a prediction modeapplied to the macroblocks is a predetermined prediction mode; anencoded data selection step for selecting a predetermined type ofencoded data corresponding to the pixels at predetermined positions inthe macroblocks to which the predetermined prediction mode is applied;and a bit distribution step for more preferentially distributing bits toa predetermined type of encoded data selected at the encoded dataselection step than a predetermined type of encoded data that is notselected at the encoded data selection step in the macroblocks to whichthe predetermined prediction mode is applied.

In the encoded data output method, the bit distribution step may reducethe predetermined type of encoded data which is not selected at theencoded data selection step.

In the encoded data output method, the bit distribution step maydistribute bits as many as or less than the bits in the input bit streamto the predetermined type of encoded data selected at the encoded dataselection step.

In the encoded data output method, the predetermined prediction mode maybe a mode for further dividing the submacroblocks constituting each ofthe macroblocks to one or a plurality of secondary submacroblocks andarranging the secondary submacroblocks as motion compensation blocks.

In the encoded data output method, the predetermined positions may befour corners of the macroblocks.

In the encoded data output method, when the predetermined type ofencoded data of other macroblock lacks, the predetermined positions maybe positions relating to the predetermined type of encoded data utilizedto reproduce the predetermined type of lacking encoded data.

In the encoded data output method, the predetermined type of encodeddata may include at least a motion vector.

In the encoded data output method, a predicting direction determinationstep may be applied to determine whether or not a predicting directionapplied to the respective submacroblocks included in the macroblocks, towhich the predetermined prediction mode is applied, is bidirection; andthe encoded data selection step may be applied only to thesubmacroblocks to which it is determined that the predicting directionthereof is not bidirection at the predicting direction determinationstep.

In the encoded data output method, a predicting direction determinationstep may be applied to determine whether or not a predicting directionapplied to the respective submacroblocks included in the macroblocks, towhich the predetermined prediction mode is applied, is bidirection; andthe encoded data selection step may be applied to the submacroblocks towhich it is determined that the predicting direction thereof is notbidirection at the predicting direction determination step, and theencoded data selection step is applied or not applied to thesubmacroblocks to which it is determined that the predicting directionthereof is bidirection at the predicting direction determination step.

In the encoded data output method, a predicting direction determinationstep may be applied to determine whether or not a predicting directionapplied to the respective submacroblocks included in the macroblocks, towhich the predetermined prediction mode is applied, is bidirection; andthe encoded data selection step may be applied to the macroblocks onlywhen the number of the submacroblocks, which is included in themacroblocks to which the predetermined prediction mode is applied, andto which a bidirectional predicting direction is applied, is apredetermined value or less.

In the encoded data output method, the encoded data selection step mayselect a predetermined type of encoded data, which corresponds tosecondary submacroblocks including the pixels at the predeterminedpositions in the macroblocks, in the macroblocks to which thepredetermined prediction mode is applied.

In the encoded data output method, bits may not be reduced as to encodeddata of a type other than the predetermined type.

According to a second aspect of the present invention, there is provideda recoded data creation method including the respective steps of theencoded data output method; a decode step for decoding a base bandsignal from an input bit stream; and a multiplex step for multiplexingencoded data, which is obtained by applying the encoded data outputmethod to the input bit stream, to the base band signal.

According to a third aspect of the present invention, there is providedan encoded data restore method of restoring the encoded data ofrespective macroblocks for encoding based on the encoded data ofrespective macroblocks created by an encoded data output method ofoutputting encoded data of respective macroblocks corresponding to aninput bit stream in a predetermined number of bits or less, the methodincluding a prediction mode determination step for determining whetheror not a prediction mode applied to the macroblocks is a predeterminedprediction mode; an encoded data selection determination step fordetermining, in the encoded data output method, that encoded data of apredetermined type corresponding to the pixels at the predeterminedpositions in the macroblocks is selected in the macroblocks to which thepredetermined prediction mode is applied; a bit distributiondetermination step for determining, in the encoded data output method,that bits are more preferentially distributed to a predetermined type ofencoded data selected at the encoded data selection step than apredetermined type of encoded data that is not selected at the encodeddata selection step in the macroblocks to which the predeterminedprediction mode is applied; and an encoded data restore step forrestoring the encoded data of the respective macroblocks for recodingbased on a result of the bit distribution determination step.

In the encoded data restore method, it may be determined at the bitdistribution determination step that the predetermined type of encodeddata, which is determined to be not selected in the encoded data outputmethod at the encoded data selection determination step, is reduced inthe encoded data output method.

In the encoded data restore method, it may be determined at the bitdistribution determination step that bits, which are the same or lessthan the bits in the input bit stream, are distributed to or reducedfrom the predetermined type of encoded data which is determined to beselected in the encoded data output method at the encoded data selectiondetermination step in the encoded data output method.

In the encoded data restore method, the predetermined prediction modemay be a mode for further dividing the submacroblocks constituting eachof the macroblocks to one or a plurality of secondary submacroblocks andarranging the secondary submacroblocks as motion compensation blocks.

In the encoded data restore method, the predetermined positions may befour corners of the macroblocks.

In the encoded data restore method, when the predetermined type of codeddata of other macroblock lacks, the predetermined positions may bepositions relating to the predetermined type of coded data utilized toreproduce the predetermined type of lacking coded data.

In the encoded data restore method, the predetermined type of coded datamay include at least a motion vector.

In the encoded data restore method, a predicting direction determinationstep may be applied to determine whether or not a predicting directionapplied to the respective submacroblocks included in the macroblocks, towhich the predetermined prediction mode is applied, is bidirection; andthe encoded data selection step may be applied only to thesubmacroblocks to which it is determined that the predicting directionthereof is not bidirection at the predicting direction determinationstep.

In the encoded data restore method, a predicting direction determinationstep may be applied to determine whether or not a predicting directionapplied to the respective submacroblocks included in the macroblocks, towhich the predetermined prediction mode is applied, is bidirection; andthe encoded data selection step may be applied to the submacroblocks towhich it is determined that the predicting direction thereof is notbidirection at the predicting direction determination step, and theencoded data selection step is applied or not applied to thesubmacroblocks to which it is determined that the predicting directionthereof is bidirection at the predicting direction determination step.

In the encoded data restore method, a predicting direction determinationstep may be applied to determine whether or not a predicting directionapplied to the respective submacroblocks included in the macroblocks, towhich the predetermined prediction mode is applied, is bidirection; andthe encoded data selection step may be applied to the macroblocks onlywhen the number of the submacroblocks, which is included in themacroblocks to which the predetermined prediction mode is applied, andto which a bidirectional predicting direction is applied, is apredetermined value or less.

In the encoded data restore method, at the encoded data selection step,it may be determined that a predetermined type of encoded data, whichcorresponds to secondary submacroblocks including the pixels at thepredetermined positions in the macroblocks, is selected in themacroblocks to which the predetermined prediction mode is applied, inthe encoded data output method.

In the encoded data restore method, as to a type of encoded data otherthan the predetermined type, the encoded data may be restored based onthat bits are not reduced in the encoded data output method.

According to a fourth aspect of the present invention there is providedan encoding method including the respective steps of the encoded datarestore method; and an encode step for encoding an input base bandsignal making use of the encoded data restored by the encoded datarestore method.

The encoded data output method according to the present inventionincludes the prediction mode determination step for determining whetheror not a prediction mode applied to the macroblocks is a predeterminedprediction mode, the encoded data selection step for selecting apredetermined type of encoded data corresponding to the pixels atpredetermined positions in the macroblocks to which the predeterminedprediction mode is applied, and the bit distribution step for morepreferentially distributing bits to a predetermined type of encoded dataselected at the encoded data selection step than a predetermined type ofencoded data that is not selected at the encoded data selection step inthe macroblocks to which the predetermined prediction mode is applied.Accordingly, it is possible to suppress the number of bits of theencoded data per each macroblock to less than a predetermined value aswell as to limit the encoded data per one macroblock transmitted fromthe decoding device to the recoding device to less than a predeterminedvalue while suppressing the lack ratio of the amount of information ofthe encoded data as much as possible by properly setting a predeterminedprediction mode, and a predetermined type of encoded data correspondingto the pixels at the predetermined positions in the macroblocks or acombination of them.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A to 1D are views showing a manner for dividing a macroblock tofour types;

FIGS. 2A to 2D are views showing a manner for dividing a submacroblockto four types;

FIG. 3 is a table for explaining the number of bits of encoded data perone macroblock in a conventional example;

FIG. 4 is a block diagram showing a structure of an image decodingdevice with an encoded data multiplexing function according to anembodiment of the present invention;

FIG. 5 is a flowchart showing a flowchart showing an encoded dataselection method carried out by an encoded data selection unit shown inFIG. 4;

FIGS. 6A to 6F are views showing a motion vector transmitted when theencoded data selection method shown in FIG. 5 is not applied in anexample 1 of present invention;

FIG. 7 is a view showing pixels to be included in a secondarysubmacroblock (that is, motion compensation block) by which a motionvector is selected in the encoded data selection method shown in FIG. 5;

FIGS. 8A to 8D are views showing a secondary submacroblock (that is,motion compensation block) including the pixels shown in FIG. 7 in theencoded data selection method shown in FIG. 5;

FIGS. 9A to 9F are views showing a motion vector selected by the encodeddata selection method shown in FIG. 5 in the example 1 of the presentinvention;

FIG. 10 is a table showing the secondary submacroblock (that is, amotion compensation block) in which a motion vector is selected by theencoded data selection method shown in FIG. 5 in the example 1 of thepresent invention;

FIG. 11 is a table explaining the number of bits of encoded data per onemacroblock when the encoded data selection method shown in FIG. 5 isapplied in the example 1 of the present invention;

FIG. 12 is a view explaining a direct mode in H.264;

FIG. 13 is a block diagram showing an arrangement of an image recodingdevice according to the embodiment of the present invention;

FIG. 14 is a flowchart showing an encoded data setting method carriedout by an encoded data setting unit shown in FIG. 13;

FIGS. 15A to 15F are views showing a motion vector selected by theencoded data selection method shown in FIG. 5 in an example 2 of thepresent invention;

FIG. 16 is a table showing the secondary submacroblock (that is, motioncompensation block) in which a motion vector is selected by the encodeddata selection method shown in FIG. 5 in an example 2 of the presentinvention; and

FIG. 17 is a table explaining the number of bits of encoded data per onemacroblock when the encoded data selection method shown in FIG. 5 isapplied in the example 2 of the present invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

A best mode for embodying the present invention will be explained belowin detail with reference to the drawings.

EXAMPLE 1

Referring to FIG. 4, an image decoding device with an encoded datamultiplexing function according to the embodiment has an image decodingunit 100, a multiplexer 110, and an encoded data selection unit 120 as acharacteristic unit of the present invention.

The image decoding unit 100 includes an entropy decoding unit 101, aninverse quantization/inverse transformation unit 102, an image buffer103, and a prediction unit 104. Operation of the image decoding unit 100will be explained below.

The entropy decoding unit 101 extracts coefficient information andprediction information by entropy decoding a bit stream input thereto.

The inverse quantization/inverse transformation unit 102 obtains aprediction error by inverse quantizing and inverse transforming thecoefficient information supplied from the entropy decoding unit 101.

The prediction unit 104 creates a predicted image from a reconstructedimage stored in the image buffer 103 making use of the predictioninformation supplied from the entropy decoding unit 101.

The predicted image is made to the reconstructed image by being addedwith the prediction error supplied from the inverse quantization/inversetransformation unit 102. The reconstructed image is stored in the imagebuffer 103 so that it is decoded later.

The reconstructed image stored in the image buffer 103 is output to theoutside through the multiplexer 110 at appropriate display timing.

The image decoding unit 100 has been explained as above.

Subsequently, operation of the multiplexer 110 will be explained.

The multiplexer 110 has a function for multiplexing encoded data, whichis extracted and formatted by the encoded data selection unit 120, to avideo base band signal decoded by the image decoding unit 100. The videobase band signal is synchronized with the encoded data. The format afterthe multiplexing is carried out is prescribed by SMPTE 319-2000, andmultiplexing is carried out at chroma (LSB) of a video signal havingluminance of 10 bits and chroma of 10 bits.

The multiplexer 110 has been explained as above.

Subsequently, the encoded data selection unit 120 as the characteristicunit of the present invention will be explained.

The encoded data is supplied to the encoded data selection unit 120 fromthe entropy decoding unit 101 in the image decoding device.

Operation of the encoded data selection unit 120 will be explained withreference to a flowchart of FIG. 5.

At step S151, it is determined whether or not mb_type is the intra-mode.When mb_type is the intra-mode, the process the process goes to stepS153, and when mb_type is not the intra-mode, the process goes to stepS152.

At step S152, it is determined whether or not mb_type is a tree-mode.The tree mode is a mode in which a macroblock is divided into foursubmacroblocks as shown in FIG. 1D and further each submacroblock isdivided into a secondary submacroblock shown in any of FIGS. 2A to FIG.2D. When mb_type is not the tree mode, the process goes to step S154,and when mb_type is the tree mode, the process goes to step S155.

At step S153 (intra-mode), since the encoded data input from the entropydecoding unit 101 has 87 bits per one macroblock as shown in FIG. 3 andthus has bits less than 256 bits, the encoded data is output as it is.That is, all the encoded data is selected. Accordingly, in theintra-mode, the information of the encoded data does not lack. After theencoded data is selected, the processing is finished (prepares forselection of the encoded data of a next macroblock).

As apparent from FIG. 3, at step 154 (at which mb_type is the inter-modebut is not the in tree mode) (inter 16×16, 16×8, or 8×16), the code dataper one macroblock can be expressed by:

-   52 bits as to encoded data of 16×16 mode when it is predicted in one    direction and 83 bits when it is predicted in bidirection;-   83 bits as to encoded data of 16×8 mode or 8×16 mode predicted only    in one direction;-   114 bits as to encoded data of 16×8 mode or 8×16 mode when one of    them is predicted in one direction and the other is predicted in    bidirection; and-   145 bits as to encoded data of 16×8 mode or 8×16 mode predicted only    in bidirection.

Since the number of bits of the code data per one macroblock is lessthan 256 bits, the encoded data is output as it is. That is, all theencoded data is selected. Accordingly, in the mode which is theinter-mode but is not the tree mode, the information of the encoded datadoes not lack. After the encoded data is selected, the processing isfinished (prepares for selection of the encoded data of a nextmacroblock).

At step S155 (tree mode), a macroblock is classified to six typesreferring to the encoded data as shown in FIGS. 6A to 6F.

The type shown FIG. 6A is a type of a macroblock in a P picture and all(four) the submacroblocks are predicted in forward direction. Since eachsubmacroblock is divided to one to four secondary submacroblocks, thenumber of motion compensation blocks per one macroblock is 4 to 16. FIG.6A shows an example in which the number of motion compensation blocksper one macroblock is 9. Further, the number of motion vectors per onemotion compensation block is 1 at all times. Accordingly, in the typeshown in the FIG. 6A, the maximum number of motion vectors per onemacroblock is 16. As shown in a seventh column of FIG. 3, the encodeddata per one macroblock of the type shown in FIG. 6A has 473 bits at themaximum.

The type shown in FIG. 6B is one type of a plurality of types of amacroblock in a B picture, and all (four) the submacroblocks arepredicted in forward direction or backward direction. Since eachsubmacroblock is divided to one to four secondary submacroblocks, thenumber of motion compensation blocks per one macroblock is 4 to 16. FIG.6A shows an example in which the number of motion compensation blocksper one macroblock is 9. Further, the number of motion vectors per onemotion compensation block is 1 at all times. Accordingly, in the typeshown in FIG. 6B, the maximum number of motion vectors per onemacroblock is 16. As shown in an eighth column of FIG. 3, the encodeddata per one macroblock of the type shown in FIG. 6B has 473 bits at themaximum.

The type shown in FIG. 6C is one type of the plurality of types of themacroblock in the B picture, and three submacroblocks are predicted inforward direction or backward direction and one submacroblock ispredicted in bidirection. Since each submacroblock is divided to one tofour secondary submacroblocks, the number of motion compensation blocksper one macroblock is 5 to 20. FIG. 6C shows an example in which thenumber of motion compensation blocks per one macroblock is 9. Further,the number of motion vectors per one motion compensation block predictedin forward direction or backward direction is 1 at all times, and thenumber of motion vectors per one motion compensation block predicted inbidirection is 2 at all times. Accordingly, in the type shown in FIG.6C, the maximum number of motion vectors per one macroblock is 20. Asshown in a ninth column of FIG. 3, the encoded data per one macroblockof the type shown in FIG. 6C has 582 bits at the maximum.

The type shown in FIG. 6D is one type of the plurality of types of themacroblock in the B picture, two submacroblocks are predicted in forwarddirection or backward direction, and two submacroblock are predicted inbidirection. Since each submacroblock is divided to one to foursecondary submacroblocks, the number of motion compensation blocks perone macroblock is 6 to 24. The FIG. 6D shows an example in which thenumber of motion compensation blocks per one macroblock is 9. Further,the number of motion vectors per one motion compensation block predictedin forward direction or backward direction is 1 at all times, and thenumber of motion vectors per one motion compensation block predicted inbidirection is 2 at all times. Accordingly, in the type shown in FIG.6D, the maximum number of motion vectors per one macroblock is 24. Asshown in a tenth column of FIG. 3, the encoded data per one macroblockof the type shown in FIG. 6D has 691 bits at the maximum.

The type shown in FIG. 6E is one type of the plurality of types of themacroblock in the B picture, one submacroblock is predicted in forwarddirection or backward direction, and three submacroblocks are predictedin bidirection. Since each submacroblock is divided to one to foursecondary submacroblocks, the number of motion compensation blocks perone macroblock is 7 to 28. The FIG. 6E shows an example in which thenumber of motion compensation blocks per one macroblock is 9. Further,the number of motion vectors per one motion compensation block predictedin forward direction or backward direction is 1 at all times, and thenumber of motion vectors per one motion compensation block predicted inbidirection is 2 at all times. Accordingly, in the type shown in FIG.6E, the maximum number of motion vectors per one macroblock is 28. Asshown in an eleventh column of FIG. 3, the encoded data per onemacroblock of the type shown in the FIG. 6E has 800 bits at the maximum.

The type shown in FIG. 6F is one type of the plurality of types of themacroblock in the B picture, and all (four) the submacroblocks arepredicted in bidirection. Since each submacroblock is divided to one tofour secondary submacroblocks, the number of motion compensation blocksper one macroblock is 8 to 32. The FIG. 6F shows an example in which thenumber of motion compensation blocks per one macroblock is 9. Further,the number of motion vectors per one motion compensation block predictedin bidirection is 2 at all times. Accordingly, in the type shown in FIG.6F, the maximum number of motion vectors per one macroblock is 32. Asshown in a twelfth column of FIG. 3, the encoded data per one macroblockof the type shown in FIG. 6F has 909 bits at the maximum.

Accordingly, the maximum number of encoded bits per one macroblockexceeds 256 bits in any of the six types shown in FIGS. 6A to 6F asshown in the seventh to twelfth columns of FIG. 3.

To cope with this problem, at step S155, the number of bits of theencoded data per one macroblock is set less than 256 bits by reducingthe number of motion vectors per one macroblock of the tree mode.

In the submacroblock predicted in forward direction or backwarddirection, the number of motion vectors per submacroblock is 4 at themaximum. In the embodiment, however, a sufficient number of bits areallocated only to one motion vector and no number of bits are allocatedto the other motion vectors. For example, only one motion vector isselected.

In contrast, in the submacroblock predicted in bidirection, although thenumber of motion vectors per submacroblock is 8 at the maximum, nonumber of bits are allocated to all the motion vectors. That is, nomotion vector is selected.

Further, a motion vector as to a motion compensation block including thepixels at the four corners of a macroblock shown in FIG. 7 is used asone motion vector selected with respect to one submacroblock predictedin forward direction or backward direction. That is, to explain, forexample, the upper left submacroblock of four submacroblocks shown inFIG. 7, when it is composed of only one secondary submacroblock (thatis, motion compensation block) with 8×8 pixels as shown in FIG. 8A, amotion vector as to the secondary submacroblock (that is, motioncompensation block) is used as one motion vector selected with respectto a macroblock. When the submacroblock is composed of two secondarysubmacroblock (that is, motion compensation blocks) of 8×4 pixels asshown in FIG. 8B, a motion vector as to an upper secondary submacroblock(that is, motion compensation block) is used as one motion vectorselected with respect to a macroblock. When the submacroblock iscomposed of two secondary submacroblocks (that is, motion compensationblocks) of 4×8 pixels as shown in FIG. 8C, a motion vector as to a leftsecondary submacroblock (that is, motion compensation block) is used asone motion vector selected with respect to a macroblock. When thesubmacroblock is composed of four secondary submacroblocks (that is,motion compensation blocks) of 4×4 pixels as shown in FIG. 8D, a motionvector as to an upper left secondary submacroblock (that is, motioncompensation block) is used as one motion vector selected with respectto a macroblock. Although explanation of lower left, upper right, andlower right submacroblocks in the four submacroblocks shown in FIG. 7 isomitted, when, for example, the lower left submacroblock is composed offour secondary submacroblocks (that is, motion compensation blocks) of4×4 pixels, a motion vector as to the lower left secondary submacroblock(that is, motion compensation block) is used as one motion vectorselected with respect to the lower left macroblock.

Which of the secondary submacroblocks includes the pixels at the fourcorners as shown in FIG. 7 can be determined based on the encoded data.

FIGS. 9A to 9F show examples of a case according to the above selectionmethod. They correspond to the examples shown in FIGS. 6A to 6F.

In the type shown in FIG. 6A, since all (four) the submacroblocks arepredicted in forward direction, one motion vector is selected as to eachof the four submacroblocks as shown in FIG. 9A. Thus, four motionvectors are selected per one macroblock. Further, the selected motionvectors are motion vectors as to a secondary submacroblock (that is,motion compensation block) including pixels shown by “x” in of FIG. 9A.

In the type shown in FIG. 6B, since all (four) the submacroblocks arepredicted in forward direction or backward direction, one motion vectoris selected as to each of the respective submacroblocks as shown in FIG.9B. Thus, four motion vectors are selected per one macroblock. Further,the selected motion vectors are motion vectors as to a secondarysubmacroblock (that is, motion compensation block) including pixelsshown by “x” in FIG. 9B.

In the type shown in FIG. 6C, since three submacroblocks are predictedin forward direction or backward direction and one submacroblock ispredicted in bidirection, one motion vector is selected as to each ofthe three submacroblocks predicted in forward direction or backwarddirection and a motion vector is discarded as to the one submacroblockpredicted in bidirection as shown in FIG. 9C. Thus, three motion vectorsare selected per one macroblock. Further, the selected motion vectorsare macroblocks as to a secondary submacroblock (that is, motioncompensation block) including pixels shown by “x” in FIG. 9A.

In the type shown in FIG. 6D, since two submacroblocks are predicted inforward direction or backward direction and two submacroblocks arepredicted in bidirection, one motion vector is selected as to each ofthe two submacroblocks predicted forward or backward direction and amotion vector is discarded as to the two submacroblocks predicted inbidirection as shown in FIG. 9D. Thus, two motion vectors are selectedper one macroblock. Further, the selected motion vectors are macroblocksas to a secondary submacroblock (that is, motion compensation block)including pixels shown by “x” in FIG. 9D.

In the type shown in FIG. 6E, since one submacroblock is predicted inforward direction or backward direction and three submacroblocks arepredicted in bidirection, one motion vector is selected as to each ofthe three submacroblocks predicted in forward direction or backwarddirection and a motion vector is discarded as to the three submacroblockpredicted in bidirection as shown in FIG. 9E. Thus, one motion vector isselected per one macroblock. Further, the selected motion vectors aremacroblocks as to a secondary submacroblock (that is, motioncompensation block) including pixels shown by “x” in FIG. 9E.

In the type shown in FIG. 6F, since all (four) the submacroblocks arepredicted in bidirection, a motion vector is discarded as to the foursubmacroblocks predicted in bidirection as shown in FIG. 9F. Thus, nomotion vector is selected per one macroblock.

FIG. 10 shows a table showing whether or not a motion vector ofrespective motion compensation blocks is selected in the respectivecases of FIGS. 9A to 9F.

Note that, as apparent when FIGS. 6A and 6B are compared with FIGS. 9Aand 9B, when a certain submacroblock (for example, upper leftsubmacroblocks shown in FIGS. 6A and 6B) is composed of one secondarysubmacroblock with 8×8 pixels as well as predicted in forward directionor backward direction, the number of motion vectors corresponding to thesubmacroblock is one from the beginning and thus the one motion vectoris inevitably selected. As a result, all the motion vectorscorresponding to the one secondary submacroblock with 8×8 pixelsincluded in the submacroblock are selected.

As apparent from the above explanation, in the macroblock of the treemode, information lacks in a manner described follows.

The motion vectors of a secondary submacroblock with 8×8 pixels (thatis, motion compensation block) predicted in one direction are selected.

Further, in the secondary submacroblocks with pixels less than 8×8pixels (that is, motion compensation blocks) (specifically, secondarysubmacroblock with 8×4 pixels predicted in one direction, secondarysubmacroblock with 4×8 pixels predicted in one direction, and secondarysubmacroblock with 4×4 pixels predicted in one direction), the motionvectors of secondary submacroblocks with the pixels at the four cornersof a macroblock (the pixels shown in FIG. 7) are selected.

On the other hand, in the secondary submacroblocks with pixels less than8×8 (that is, motion compensation blocks) (specifically, secondarysubmacroblock with 8×4 pixels predicted in one direction, secondarysubmacroblock with 4×8 pixels predicted in one direction, and secondarysubmacroblock with 4×4 pixels predicted in one direction), the motionvectors of secondary submacroblocks no pixels at the four corner of amacroblock (the pixels shown in FIG. 7) are discarded.

Further, the motion vectors of all the secondary submacroblocks (thatis, motion compensation blocks) included in the submacroblock predictedin bidirection are selected.

Note that all the encoded data other than the motion vectors areselected. Accordingly, although ref_idx is entirely transmitted, it maynot be transmitted.

When the motion vectors are selected by the selection method, the numberof bits of the encoded data per one macroblock is as shown in FIG. 11.FIG. 11 will be briefly described from a left side as follows:

-   (1) Intra-mode: 87 bits;-   (2) Inter-mode, first macroblock type, predicted in forward    direction or backward direction: 52 bits;-   (3) Inter-mode, first macroblock type, predicted in bidirection: 83    bits;-   (4) Inter-mode, second or third macroblock type, two submacroblocks    are predicted in forward direction or, backward direction: 83 bits;-   (5) Inter-mode, second or third macroblock type, one submacroblock    is predicted in forward direction or backward direction, the other    one submacroblock is predicted in bidirection: 114 bits;-   (6) Inter-mode, second or third macroblock type, two submacroblocks    are predicted in bidirection: 145 bits;-   (7) Inter-mode, first to fourth macroblock types, all the motion    compensation blocks are predicted in forward direction in P picture:    161 bits;-   (8) Inter-mode, fourth macroblock type, four submacroblocks are    predicted in forward direction or backward direction: 161 bits;-   (9) Inter-mode, fourth macroblock type, three submacroblocks are    predicted in forward direction or backward direction, one    submacroblock is predicted in bidirection: 140 bits;-   (10) Inter-mode, fourth macroblock type, two submacroblocks are    predicted in forward direction or backward direction, two    submacroblocks are predicted in bidirection: 119 bits;-   (11) Inter-mode, fourth macroblock type, one submacroblock is    predicted in forward or backward direction, three submacroblocks are    predicted in bidirectional prediction: 98 bits; and-   (12) Inter-mode, fourth macroblock type, four submacroblocks are    predicted in bidirection: 77 bits Accordingly, even if a macroblock    has any type, the number of bits of the encoded data per one    macroblock can be suppressed to 256 bits or less.

Note that, in the cases of the items (1) to (6), since the number ofbits of the encoded data per one macroblock is 256 bits or less from thebeginning, bits are not reduced at step S153 or S154 carried out inthese cases, and thus the numbers of the encoded data per one macroblockis the same as the values shown in FIG. 3. In the cases of the items (7)to (12), since the number of bits of the encoded data per one macroblockexceeds 256 bits from the beginning, bits are reduced at step S155carried out in these cases, and thus the number of the encoded data perone macroblock is reduced from the values shown in FIG. 3.

It is possible to restrict the maximum number of bits per one macroblockto a predetermined value (for example, 256 bits) or less by setting alimit to the number of motion vectors making use of the selection methoddescribed above. However, the frequency of occurrence of the secondarysubmacroblocks with a size less than 8×8 bits to which the selection ofmotion vector is applied (that is, motion compensation blocks) isstatistically small, the encoded data has a small lack ratio.Accordingly, even if an image is recoded making use of the encoded datausing the selection method described above, it is possible to suppressdeterioration of image quality of a recoded image as much as possible.

In the submacroblocks of the tree mode of the embodiment, one motionvector located at the four corners of the macroblock is selected as arepresentative motion vector of the submacroblock as shown in FIG. 7making use of the following three characteristics.

-   Frequency of occurrence of the tree mode is statistically low;-   Frequency of occurrence of secondary submacroblocks (that is, motion    compensation blocks) with pixels less than 8×8 bits is statistically    low in the tree mode; and-   Frequency of occurrence of a direct mode, which can reproduce motion    vector from the P picture at the same position, is high in the B    picture having a tendency to have many motion vectors

As shown in FIG. 12, the motion vectors located at the four corners ofthe macroblock are motion vectors used by the direct mode of H.264 (asto the detail of the direct mode, refer to the advice of H.264, ISO/IEC14496-10 8.4.1.2 Derivation process for luma motion vectors for B_Skip,B_Direct_(—)16×16, and B_Direct_(—)8×8 ). In the direct mode, the motionvector is calculated based on the motion vector as to the position shownin FIG. 12. Accordingly, even if the motion vector of the B picture isnot selected and lacks, it can be reproduced from the motion vector ofan adjacent reference picture (shown by “Reference PIC1” in FIG. 12) ina time axis.

FIG. 13 shows an image recoding device to which data, which is obtainedby multiplexing a video base band signal output from the image decodingdevice with the encoded data multiplexing function shown in FIG. 4 toencoded data, is input. As described above, the amount of information ofencoded data (specifically, motion vector of 1 or more) lacks in a partof macroblock in the tree mode. Accordingly, the recoding device must beoperated in consideration of it.

The image recoding device of the embodiment includes a demultiplexer210, an image encoding unit 200, and an encoded data setting unit 220 asa characteristic unit of the present invention.

The demultiplexer 210 will be explained.

The demultiplexer 210 carries out operation opposite to the multiplexer110. That is, the demultiplexer 210 is input with the video base bandsignal to which the encoded data is multiplexed and separates them fromeach other. The format of the multiplexed encoded data is prescribed bySMPTE 319-2000 and a multiplexing position is the chroma LSB of a videosignal having luminance of 10 bits and chroma of 10 bits.

The demultiplexer 210 has been explained as above.

Subsequently, the image encoding unit 200 will be explained.

The image encoding unit 200 includes a macroblock buffer (MB buffer)201, a transformation/quantization unit 202, an entropy coding unit 203,an inverse quantization/inverse transformation unit 204, an image buffer205, a prediction unit 206, a prediction data estimation unit 207, and aswitch 208.

The MB buffer 201 stores an image of one macroblock supplied from thedemultiplexer 210.

The prediction data estimation unit 207 estimates prediction informationfor preferably predicting an input image stored in the MB buffer 201from a reconstructed image stored in the image buffer 205.

The prediction unit 206 creates a predicted image from the reconstructedimage stored in the image buffer 205 making use of the predictioninformation supplied through the switch 208.

A prediction error is obtained by subtracting the predicted image, whichis supplied from the prediction unit 206, from the image stored in theMB buffer 201.

The prediction error is transformed and quantized by thetransformation/quantization unit 202 and coefficient information as anoutput therefrom is supplied to the entropy coding unit 203 and theinverse quantization/inverse transformation unit 204.

The entropy coding unit 203 makes the coefficient information and theprediction information supplied thereto to an entropy code and outputsthe coded train thereof to the outside as a bit stream.

The inverse quantization/inverse transformation unit 204 restores aprediction error by inverse quantizing and inverse transforming thecoefficient information supplied from the transformation/quantizationunit 202.

The predicted image is made to a reconstructed image by being added withthe prediction error supplied from the inverse quantization/inversetransformation unit 204. The reconstructed image is stored in the imagebuffer 205 so that it is encoded later.

A bit stream is obtained by applying the above processing to themacroblocks of all the images.

The image encoding unit 200 has been explained as above.

Subsequently, the the encoded data setting unit 220 as thecharacteristic unit of the present invention will be explained.

The encoded data setting unit 220 is supplied with encoded data from thedemultiplexer 210. The encoded data setting unit 220 controls the switch208 according to the encoded data input thereto. When the amount of theinformation of the encoded data lacks, prediction information of theprediction data estimation unit 207 is, supplied to the prediction unit206, whereas when it does not lack, the prediction information includedin the encoded data supplied from the demultiplexer 210 is supplied tothe prediction unit 206.

Operation of the encoded data selection unit 220 and the switch 208 willbe explained with reference to a flowchart of FIG. 14.

At step S15, it is determined whether or not mb_type is the intra-mode.When mb_type is the intra-mode, the process goes to step S253, and whenmb_type is the intra-mode, the process goes to step S252.

At step S252, it is determined whether or not mb_type is the the treemode. When mb_type is not the tree mode, the process goes to step, andwhen mb_type is the tree mode, the process goes to step S255.

At step S253 (intra-mode), since the encoded data of intra-mode can beshown by a number of bits smaller than 256 bits of the transmission bandfrom the beginning as apparent from the explanation of the imagedecoding device with the encoded data multiplexing function, the amountof information is not reduced at all by the encoded data selection unit120. Accordingly, the prediction information included in the encodeddata supplied from the demultiplexer 210 is supplied to the predictionunit 206 by switching the switch 208 to the encoded data setting unit220 side, thereby the processing is finished.

At step S254 (at which mb_type is the inter-mode but is not the treemode) (inter 16×16, 16×8, or 8×16), the encoded data of the macroblockof the mode which is the inter-mode but is not the tree mode from thebeginning can be shown by a transmission band with bits smaller than 256bits as apparent from explanation of the image decoding device with theencoded data multiplexing function. Thus, the amount of information isnot reduced at all by the encoded data selection unit 120. Accordingly,the prediction information included in the encoded data supplied fromthe demultiplexer 210 is supplied to the prediction unit 206 byswitching the switch 208 to the encoded data setting unit 220 side,thereby the processing is finished.

How information lacks in the macroblock of the tree mode has beenexplained in the explanation of the image decoding device with theencoded data multiplexing function. In accordance with the lack of theinformation, at step S255 (tree mode), as to the secondary submacroblock(that is, motion compensation block) whose information (specifically,motion vector) lacks, the prediction information, which includes themotion vector estimated by the prediction data estimation unit 207, issupplied to the prediction unit 206. On the other hand, as to thesecondary submacroblock (that is, motion compensation block) whoseinformation (specifically, motion vector) is selected by the encodeddata selection unit 120 (does not lack), the prediction informationincluded in the encoded data supplied from the demultiplexer 210 issupplied to the prediction unit 208. Note that as to the secondarysubmacroblock whose information (specifically, motion vector) lacks(that is, motion compensation block), when the motion vector can beobtained by the direct mode, it is utilized. Further, since the encodeddata other than the motion vector is selected by the encoded dataselection unit 120, it is supplied to the prediction unit 206.

At step S255 (tree mode), the encoded data setting unit 220 determinesto which one of the six kinds of types shown in FIGS. 6A to 6F therespective macroblocks belong referring to the encoded data suppliedthereto likewise the encoded data selection unit 120. Further, theencoded data setting unit 220 determines by which one of the divisionmodes shown in FIGS. 2A to FIG. 2D the respective submacroblocks aredivided referring to the encoded data likewise the encoded dataselection unit 120. Then, it is determined based on the abovedetermination that the motion vector of which secondary submacroblock isselected by the encoded data selection unit 120 and the motion vector ofwhich secondary submacroblock is discarded thereby.

Further, when how respective encoded data are disposed in a region of256 bits per one macroblock is previously determined between the encodeddata selection unit 120 and the encoded data setting unit 220 and howmotion vectors are disposed in it is previously determined, the encodeddata setting unit 220 can find which motion vector supplied from theencoded data selection unit 120 corresponds to which secondarysubmacroblock to which it is determined that a motion vector has beenselected. When, for example, an order of secondary submacroblocksaccording to which selected motion vectors are transmitted thereto ispreviously determined as to respective cases, the encoded data settingunit 220 can find which motion vector supplied from the encoded dataselection unit 120 corresponds to which secondary submacroblock to whichit is determined that a motion vector has been selected.

Further, to set the number of bits of the encoded data per macroblock toequal to or less than 256 bits, it is sufficient to set the number ofmotion vectors per one macroblock equal to or less than 7. However, whenthe number of macroblocks per one macroblock selected by the selectionmethod described above is equal to or less than 6, the selectedmacroblocks may be transmitted a plurality of times making use of avacant region. How the motion vectors are disposed between the encodeddata selection unit 120 and the encoded data setting unit 220 ispreviously determined also in this case. With this determination,resistance against error in a transmission path, for example, can beincreased.

As to the secondary submacroblock whose information (specifically,motion vector) lacks (that is, motion compensation block), the motionvectors as to the secondary submacroblock may be interpolated based onthe motion vectors selected as to peripheral motion compensation blocksat step S255. Further, as to the secondary submacroblock whoseinformation (specifically, motion vector) lacks (that is, motioncompensation block), the motion vector selected as to the peripheralmotion compensation block may be used as an initiation value forestimating motion vectors as to the secondary submacroblock at stepS255. With this operation, the prediction data estimation unit 207 neednot to estimate the motion vector as to the secondary submacroblock(that is, motion compensation block) whose information (specifically,motion vector) lacks, thereby an amount of arithmetic operation can bereduced.

In the recoding device of the present invention, since the lackingmotion vector is compensated by the prediction data estimation unit 207,image quality is not greatly deteriorated. Further, as explained inrelation to the image decoding device with the encoded data multiplexingfunction, since the motion vector is compensated only by the modes whoseprobability of occurrence is low, image quality is least affectedthereby in its entirety.

EXAMPLE 2

In the selection method of the motion vector employed in the example 1,all the motion vectors are discarded (no macroblock is selected) as tothe submacroblock predicted in bidirection, although there are 8macroblocks at the maximum per one macroblock from the beginning. Withthis operation, it is possible to set the number of bits of the encodeddata per one macroblock to 161 at the maximum. However, when there is atleast one submacroblock predicted in bidirection (ninth to twelfthcolumns of FIG. 11), a band with 256 bits per one macroblock is notsufficiently used. On the other hand, when motion vectors aretransmitted as many as possible to the recode device, there is apossibility that the image quality of a restored image obtained from abit stream output from the recoding device can be increased. However,when one motion vector is selected as to a submacroblock predicted inforward or backward direction and two motion vectors are selected as toeach of 3 submacroblocks predicted in bidirection, the number of bits ofencoded data per one macroblock is set to 254 bits which are less than256 bits. However, when two window motion vectors are selected as toeach of the submacroblocks predicted in bidirection, the number of bitsof encoded data per one macroblock is set to 280 bits which exceed 256bits.

To cope with this problem, when the number of macroblocks predicted inbidirection in the macroblock is 1 to 3, a method of additionallyselecting a motion vector as to each of the macroblocks predicted inbidirection is employed as a motion vector selection method in theexample 2 based on the selection method described above.

FIGS. 15A to 15F show examples of a case according to the secondselection method. They correspond to the examples shown in FIGS. 6A to6F.

The motion vectors selected in the types of FIGS. 6A, 6B and 6F are asshown in FIGS. 9A, 9B and 9F in the first selection method and as shownin FIGS. 9A, 9B and 9F in the second selection method. However, sinceboth the selection methods are common to each other, duplicateexplanation is omitted.

In the type shown in FIG. 6C, since three submacroblocks are predictedin forward direction or backward direction and one submacroblock ispredicted in bidirection, one motion vector is selected as to each ofthe three submacroblocks predicted in forward direction or backwarddirection and two motion vectors are selected as to the onesubmacroblock predicted in bidirection as shown in FIG. 15C. Thus, thefive motion vectors are selected per one macroblock. Further, theselected motion vectors are macroblocks as to a secondary submacroblock(that is, motion compensation block) including pixels shown by “x” shownin FIG. 15C.

In the type shown in FIG. 6D, since two submacroblocks are predicted inforward direction or backward direction and two submacroblocks arepredicted in bidirection, one motion vector is selected as to each ofthe two submacroblocks predicted in forward direction or backwarddirection as shown in FIG. 15D and two motion vectors are selected as tothe two submacroblocks predicted in bidirection. Thus, the six motionvectors are selected per one macroblock. Further, the selected motionvectors are macroblocks as to a secondary submacroblock (that is, motioncompensation block) including pixels shown by “x” shown in FIG. 15D.

In the type shown in FIG. 6E, since one submacroblock is predicted inbidirection and three submacroblocks are predicted in forward directionor backward direction, one motion vector is selected as to the onesubmacroblock predicted in forward direction or backward direction asshown in FIG. 15E, and two motion vectors are selected as to each of thethree submacroblocks predicted in bidirection. Thus, the seven motionvectors are selected per one macroblock. Further, the selected motionvectors are a macroblock as to a secondary submacroblock (that is,motion compensation block) including pixels shown by “x” shown in FIG.15E.

FIG. 16 shows a table showing whether or not a motion vector ofrespective motion compensation blocks is selected in the respectivecases of FIGS. 15A to 15F.

Note that, as apparent when FIGS. 6A and 6B are compared with FIGS. 15Aand 15B, when a certain submacroblock (for example, upper leftsubmacroblocks in FIGS. 6A and 6B) are predicted in in forward orbackward direction, the number of motion vectors corresponding to thesubmacroblock is one from the beginning and thus the one motion vectoris inevitably selected. As a result, all the motion vectorscorresponding to the secondary submacroblock having one 8×8 pixelincluded in the macroblock are selected.

Further, as apparent when FIGS. 6C, 6D and 6E are compared with FIGS.15C, 15D and 15E, respectively, when a certain submacroblock (forexample, upper left submacroblocks shown in FIGS. 6C, 6D and 6E) iscomposed of one secondary submacroblock with 8×8 pixels and predicted inbidirection as well as the number of the submacroblocks predicted inbidirection of the four submacroblocks included in a macroblockincluding the submacroblocks is equal to or less than 3, the number ofmotion vectors corresponding to the certain submacroblock is two fromthe beginning and thus the two motion vectors are inevitably selected.As a result, all the motion vectors corresponding to the one secondarysubmacroblock with 8×8 pixels included in the certain macroblock areselected.

As apparent from the above description, in the macroblock of the treemode, information lacks in a manner described follows.

The motion vectors of a secondary submacroblock with 8×8 pixels (thatis, motion compensation block) predicted in one direction are selected.

Further, when the number of the submacroblocks included in a macroblockand predicted in bidirection is one to three, the motion vectors of thesecondary submacroblock (that is, motion compensation block) with 8×8pixels included in the macroblock and predicted in bidirection areselected.

Further, when the number of submacroblocks included in a macroblock andpredicted bidirection is one to three, the motion vectors of thesecondary submacroblocks, which include the pixels at the four corners(pixels shown in FIG. 7) of the secondary submacroblock with pixels lessthan 8×8 pixels (that is, motion compensation block) which is includedin the macroblock and predicted in bidirection (specifically, secondarysubmacroblock with 8×4 pixels predicted in one direction, secondarysubmacroblock with 4×8 pixels predicted in one direction, and secondarysubmacroblock with 4×4 pixels predicted in one direction) are selected.

On the other hand, the motion vectors of the secondary submacroblocks,which do not include the pixels at the four corners (pixels shown inFIG. 7) of a macroblock in the secondary submacroblock with pixels lessthan 8×8 pixels predicted in one direction (that is, motion compensationblock) (specifically, secondary submacroblock with 8×4 pixels predictedin one direction, secondary submacroblock with 4×8 pixels predicted inone direction, and secondary submacroblock with 4×4 pixels predicted inone direction), are discarded.

Further, when the number of submacroblocks included in a macroblock andpredicted bidirection is one to three, the motion vectors of thesecondary submacroblocks, which do not include the pixels at the fourcorners (pixels shown in FIG. 7) of the secondary submacroblock withpixels less than 8×8 pixels (that is, motion compensation block) whichis included in the macroblock and predicted in bidirection(specifically, secondary submacroblock with 8×4 pixels predicted in onedirection, secondary submacroblock with 4×8 pixels predicted in onedirection, and secondary submacroblock with 4×4 pixels predicted in onedirection) are discarded.

Further, when the number of submacroblocks included in a macroblock andpredicted bidirection is four, the motion vectors of all the secondarysubmacroblocks (that is, motion compensation blocks) included in themacroblock are discarded.

When the example 2 is explained in comparison with the example 1, allthe motion vectors selected in the example 1 are selected by the example2. In addition to the above, in the example 2, when the number ofsubmacroblocks included in a macroblock and predicted in bidirection isone to three, the motion vectors of the secondary submacroblock with 8×8pixels (that is, motion compensation block), which is included in themacroblock and predicted in bidirection, are selected. When the numberof submacroblocks included in a macroblock and predicted in bidirectionis one to three, the motion vectors of the secondary submacroblocks,which include the pixels at the four corners (pixels shown in FIG. 7) ofthe secondary submacroblock with pixels less than 8×8 pixels (that is,motion compensation block) which is included in the macroblock andpredicted in bidirection (specifically, secondary submacroblock with 8×4pixels predicted in one direction, secondary submacroblock with 4×8pixels predicted in one direction, and secondary submacroblock with 4×4pixels predicted in one direction) are selected.

When the motion vectors are selected as described above, the number ofbits of the encoded data per one macroblock is as shown in FIG. 17. FIG.17 will be briefly described from a left side as follows:

-   (1) Intra-mode: 87 bits;-   (2) Inter-mode, first macroblock type, predicted in forward or    backward direction: 52 bits;-   (3) Inter-mode, first macroblock type, predicted in bidirection: 83    bits;-   (4) Inter-mode, second or third macroblock type, two submacroblocks    are predicted in forward direction or backward direction: 83 bits;-   (5) Inter-mode, second or third macroblock type, one submacroblock    is predicted in forward direction or backward direction, the other    one submacroblock is predicted in bidirection: 114 bits;-   (6) Inter-mode, second or third macroblock type, two submacroblocks    are predicted in bidirection: 145 bits;-   (7) Inter-mode, first to fourth macroblock types, all the motion    compensation blocks are predicted in forward direction in P picture:    161 bits;-   (8) Inter-mode, fourth macroblock type, four submacroblocks are    predicted in forward direction or backward direction: 161 bits;-   (9) Inter-mode, fourth macroblock type, three submacroblocks are    predicted in forward direction or backward direction, one    submacroblock is predicted in bidirection: 192 bits-   (10) Inter-mode, fourth macroblock type, two submacroblocks are    predicted in forward direction or backward direction, two    submacroblocks are predicted in bidirection: 223 bits;-   (11) Inter-mode, fourth macroblock type, one submacroblock is    predicted in forward or backward direction, three submacroblocks are    predicted in bidirectional prediction: 254 bits; and-   (12) Inter-mode, fourth macroblock type, four submacroblocks are    predicted in bidirection: 77 bits

Accordingly, even if a macroblock has any type, the number of bits ofthe encoded data per one macroblock can be suppressed to 256 bits orless.

Further, no motion vector is selected in the item (12).

However, three submacroblocks of the four submacroblocks may beselected, and motion vectors may be selected as to the selectedsubmacroblocks likewise the item (11). Three submacroblocks located atpredetermined positions may be selected. Further, in order to selectthree submacroblocks, a trial calculation may be carried out to finddeterioration of image quality can be minimized when the motion vectorsof which submacroblock is discarded. Further, the degree ofdeterioration of image quality may be calculated on a trial base basedon the sizes of the motion vectors of respective motion compensationblocks, the. difference of the motion vectors between the respectivemotion compensation blocks, the degree of change of an image of eachmotion compensation block.

Further, in the item (12), three submacroblocks of the foursubmacroblocks may be selected, motion vectors may be selected as to theselected submacroblocks likewise the item (11), and further a motionvector may be sent as to the not selected submacroblock by the followingmethod.

As to the respective components of a motion vector, when a motion vectorof a secondary submacroblock of a not selected submacroblock includingthe pixels at the positions shown in FIG. 7 is within a range which canbe shown by a predetermined number of bits, the motion vector istransmitted as it is, and when it is not within the range, a valueshowing invalidity is transmitted in place of the motion vector. Forexample, it is assumed that the predetermined number of bits is 2 andthe range of it is from −0.25 pixel to 0.25 pixel. When the value of anactual motion vector is −0.25 pixel, 0 pixel, or 0.25 pixel, values, forexample, “00”, “01”, “10” showing the values of them are transmitted,respectively, and, when the value of the actual motion vector exceedsthe range, “11” showing invalidity is transmitted.

The method is not limited to the case of the item (12) and may be usedto utilize excessive bits in other case. In, for example, the item (10),the motion vector of other motion compensation block may be transmittedby the above method using excessive 33 bits. Further, in the item (12),the number of bits per one macroblock may be set to less than 256 pixelsby selecting two representative vectors to each of four submacroblockspredicted in bidirection, selecting eight macroblocks per onemacroblock, and transmitting the macroblocks in a number of bits smallerthan an ordinary number of bits. For example, one motion vector isordinarily expressed by 26 bits, it may be expressed by 24 bits. Then, amotion vector having an accuracy of 26 bits may be estimated from themotion vector expressed by 24 bit by the prediction data estimation unit207.

Further, when the number of macroblocks predicted in bidirection in amacroblock is one or one to two in place of one to three, a motionvector may be additionally selected to each of the macroblocks predictedin bidirection.

Note that, although the respective units of the devices shown in FIGS. 4and 13 may be realized by hardware, they may be realized by that acomputer reads and carries out a program for functioning it as theseunits.

Explanation has been made as to a motion vector with many bits, which isselected when a predetermined condition is satisfied and is discardedwhen it is not satisfied, as an example. However, encoded data otherthan the motion vector, for example, may be used as a target to beselected or to be not selected in addition to or in place of the motionvector based on a similar condition.

Further, a certain type of encoded data may be selected when a firstpredetermined condition (for example, the condition of the example 1) issatisfied and may be discarded when it is not satisfied, and anothercertain type of encoded data may be selected when a second predeterminedcondition (for example, the condition of the example 2) is satisfied andmay be discarded when it is not satisfied.

1. An encoded data output method of outputting encoded data ofrespective macroblocks corresponding to an input bit stream in apredetermined number of bits or less, comprising: a prediction modedetermination step for determining whether or not a prediction modeapplied to the macroblocks is a predetermined prediction mode; anencoded data selection step for selecting a predetermined type ofencoded data corresponding to the pixels at predetermined positions inthe macroblocks to which the predetermined prediction mode is applied;and a bit distribution step for more preferentially distributing bits toa predetermined type of encoded data selected at the encoded dataselection step than a predetermined type of encoded data that is notselected at the encoded data selection step in the macroblocks to whichthe predetermined prediction mode is applied.
 2. An encoded data outputmethod according to claim 1, wherein, the bit distribution step reducesthe predetermined type of encoded data which is not selected at theencoded data selection step.
 3. An encoded data output method accordingto claim 1, wherein the bit distribution step distributes bits as manyas or less than the bits in the input bit stream to the predeterminedtype of encoded data selected at the encoded data selection step.
 4. Anencoded data output method according to claim 1, wherein thepredetermined prediction mode is a mode for further dividing thesubmacroblocks constituting each of the macroblocks to one or aplurality of secondary submacroblocks and arranging the secondarysubmacroblocks as motion compensation blocks.
 5. An encoded data outputmethod according to claim 1, wherein the predetermined positions arefour corners of the macroblocks.
 6. An encoded data output methodaccording to claim 1, wherein, when the predetermined type of encodeddata of other macroblock lacks, the predetermined positions arepositions relating to the predetermined type of encoded data utilized toreproduce the predetermined type of lacking encoded data.
 7. An encodeddata output method according to claim 1, wherein the predetermined typeof encoded data includes at least a motion vector.
 8. An encoded dataoutput method according to claim 1, wherein: a predicting directiondetermination step is applied to determine whether or not a predictingdirection applied to the respective submacroblocks included in themacroblocks, to which the predetermined prediction mode is applied, isbidirection; and the encoded data selection step is applied only to thesubmacroblocks to which it is determined that the predicting directionthereof is not bidirection at the predicting direction determinationstep.
 9. An encoded data output method according to claim 1, wherein: apredicting direction determination step is applied to determine whetheror not a predicting direction applied to the respective submacroblocksincluded in the macroblocks, to which the predetermined prediction modeis applied, is bidirection; and the encoded data selection step isapplied to the submacroblocks to which it is determined that thepredicting direction thereof is not bidirection at the predictingdirection determination step, and the encoded data selection step isapplied or not applied to the submacroblocks to which it is determinedthat the predicting direction thereof is bidirection at the predictingdirection determination step.
 10. An encoded data output methodaccording to claim 1, wherein: a predicting direction determination stepis applied to determine whether or not a predicting direction applied tothe respective submacroblocks included in the macroblocks, to which thepredetermined prediction mode is applied, is bidirection; and theencoded data selection step is applied to the macroblocks only when thenumber of the submacroblocks, which is included in the macroblocks towhich the predetermined prediction mode is applied, and to which abidirectional predicting direction is applied, is a predetermined valueor less.
 11. An encoded data output method according to claim 1, whereinthe encoded data selection step selects a predetermined type of encodeddata, which corresponds to secondary submacroblocks including the pixelsat the predetermined positions in the macroblocks, in the macroblocks towhich the predetermined prediction mode is applied.
 12. An encoded dataoutput method according to claim 1, wherein bits are not reduced as toencoded data of a type other than the predetermined type.
 13. A recodeddata creation method comprising: the respective steps of the encodeddata output method according to any one of claims 1 to 12; a decode stepfor decoding a base band signal from an input bit stream; and amultiplex step for multiplexing encoded data, which is obtained byapplying the encoded data output method to the input bit stream, to thebase band signal.
 14. An encoded data restore method of restoring theencoded data of respective macroblocks for encoding based on the encodeddata of respective macroblocks created by an encoded data output methodof outputting encoded data of respective macroblocks corresponding to aninput bit stream in a predetermined number of bits or less, the methodcomprising: a prediction mode determination step for determining whetheror not a prediction mode applied to the macroblocks is a predeterminedprediction mode; an encoded data selection determination step fordetermining, in the encoded data output method, that encoded data of apredetermined type corresponding to the pixels at the predeterminedpositions in the macroblocks is selected in the macroblocks to which thepredetermined prediction mode is applied; a bit distributiondetermination step for determining, in the encoded data output method,that bits are more preferentially distributed to a predetermined type ofencoded data selected at the encoded data selection step than apredetermined type of encoded data that is not selected at the encodeddata selection step in the macroblocks to which the predeterminedprediction mode is applied; and an encoded data restore step forrestoring the encoded data of the respective macroblocks for recodingbased on a result of the bit distribution determination step.
 15. Anencoded data restore method according to claim 14, wherein it isdetermined at the bit distribution determination step that thepredetermined type of encoded data, which is determined to be notselected in the encoded data output method at the encoded data selectiondetermination step, is reduced in the encoded data output method.
 16. Anencoded data restore method according to claim 14, wherein it isdetermined at the bit distribution determination step that bits, whichare the same or less than the bits in the input bit stream, aredistributed to or reduced from the predetermined type of encoded datawhich is determined to be selected in the encoded data output method atthe encoded data selection determination step in the encoded data outputmethod.
 17. An encoded data restore method according to claim 14,wherein the predetermined prediction mode is a mode for further dividingthe submacroblocks constituting each of the macroblocks to one or aplurality of secondary submacroblocks and arranging the secondarysubmacroblocks as motion compensation blocks.
 18. An encoded datarestore method according to claim 14, wherein the predeterminedpositions are four corners of the macroblocks.
 19. An encoded datarestore method according to claim 14, wherein when the predeterminedtype of coded data of other macroblock lacks, the predeterminedpositions are positions relating to the predetermined type of coded datautilized to reproduce the predetermined type of lacking coded data. 20.An encoded data restore method according to claim 14, wherein thepredetermined type of coded data includes at least a motion vector. 21.An encoded data restore method according to claim 14 wherein: apredicting direction determination step is applied to determine whetheror not a predicting direction applied to the respective submacroblocksincluded in the macroblocks, to which the predetermined prediction modeis applied, is bidirection; and the encoded data selection step isapplied only to the submacroblocks to which it is determined that thepredicting direction thereof is not bidirection at the predictingdirection determination step.
 22. An encoded data restore methodaccording to claim 14, wherein: a predicting direction determinationstep is applied to determine whether or not a predicting directionapplied to the respective submacroblocks included in the macroblocks, towhich the predetermined-prediction mode is applied, is bidirection; andthe encoded data selection step is applied to the submacroblocks towhich it is determined that the predicting direction thereof is notbidirection at the predicting direction thereof is bidirection at thepredicting direction determination step.
 23. An encoded data restoremethod according to claim 14, wherein: a predicting directiondetermination step is applied to determine whether or not a predictingdirection applied to the respective submacroblocks included in themacroblocks, to which the predetermined prediction mode is applied, isbidirection; and the encoded data selection step is applied to themacroblocks only when the number of the submacroblocks, which isincluded in the macroblocks to which the predetermined prediction modeis applied, and to which a bidirectional predicting direction isapplied, is a predetermined value or less.
 24. An encoded data restoremethod according to claim 14, wherein, at the encoded data selectionstep, it is determined that a predetermined type of encoded data, whichcorresponds to secondary submacroblocks including the pixels at thepredetermined positions in the macroblocks, is selected in themacroblocks to which the predetermined prediction mode is applied, inthe encoded data output method.
 25. An encoded data restore methodaccording to claim 14, wherein, as to a type of encoded data other thanthe predetermined type, the encoded data is restored based on that bitsare not reduced in the encoded data output method.
 26. An encodingmethod comprising: the respective steps of the encoded data restoremethod according to any one of claims 14 to 25; and an encode step forencoding an input base band signal making use of the encoded datarestored by the encoded data restore method.
 27. An encoded data outputdevice for outputting encoded data of respective macroblockscorresponding to an input bit stream in a predetermined number of bitsor less, comprising: prediction mode determination means for determiningwhether or not a prediction mode applied to the macroblocks is apredetermined prediction mode; encoded data selection means forselecting a predetermined type of encoded data corresponding to thepixels at predetermined positions in the macroblocks to which thepredetermined prediction mode is applied; and bit distribution means formore preferentially distributing bits to a predetermined type of encodeddata selected by the encoded data selection means than a predeterminedtype of encoded data that is not selected by the encoded data selectionmeans in the macroblocks to which the predetermined prediction mode isapplied.
 28. An encoded data output device according to claim 27,wherein the bit distribution means reduces the predetermined type ofencoded data which is not selected at the encoded data selection means.29. An encoded data output device according to claim 27, wherein the bitdistribution means distributes bits as many as or less than the bits inthe input bit stream to the predetermined type of encoded data selectedby the encoded data selection means.
 30. An encoded data output deviceaccording to claim 27, wherein the predetermined prediction mode is amode for further dividing the submacroblocks constituting each of themacroblocks to one or a plurality of secondary submacroblocks andarranging the secondary submacroblocks as motion compensation blocks.31. An encoded data output device according to claim 27, wherein thepredetermined positions are four corners of the macroblocks.
 32. Anencoded data output device according to claim 27, wherein, when thepredetermined type of encoded data of other macroblock lacks, thepredetermined positions are positions relating to the predetermined typeof encoded data utilized to reproduce the predetermined type of lackingencoded data.
 33. An encoded data output device according to claim 27,wherein the predetermined type of encoded data includes at least amotion vector.
 34. An encoded data output device according to claim 27,wherein: predicting direction determination means is applied todetermine whether or not a predicting direction applied to therespective submacroblocks included in the macroblocks, to which thepredetermined prediction mode is applied, is bidirection; and theencoded data selection means is applied only to the submacroblocks towhich it is determined by the predicting direction determination meansthat the predicting direction thereof is not bidirection.
 35. An encodeddata output device according to claim 27, wherein: predicting directiondetermination means is applied to determine whether or not a predictingdirection applied to the respective submacroblocks included in themacroblocks, to which the predetermined prediction mode is applied, isbidirection; and the encoded data selection means is applied to thesubmacroblocks to which it is determined that the predicting directionthereof is not bidirection at the predicting direction determinationmeans, and the encoded data selection means is applied or not applied tothe submacroblocks to which it is determined that the predictingdirection thereof is bidirection at the predicting directiondetermination means.
 36. An encoded data output device according toclaim 27, wherein: predicting direction determination means is appliedto determine whether or not a predicting direction applied to therespective submacroblocks included in the macroblocks, to which thepredetermined prediction mode is applied, is bidirection; and theencoded data selection means is applied to the macroblocks only when thenumber of the submacroblocks, which is included in the macroblocks towhich the predetermined prediction mode is applied, and to which abidirectional predicting direction is applied, is a predetermined valueor less.
 37. An encoded data output device according to claim 27,wherein the encoded data selection means selects a predetermined type ofencoded data, which corresponds to secondary submacroblocks includingthe pixels at the predetermined positions in the macroblocks, in themacroblocks to which the predetermined prediction mode is applied. 38.An encoded data output device according to claim 27, wherein bits arenot reduced as to encoded data of a type other than the predeterminedtype.
 39. A recoded data creation device comprising: the respectivemeans of the encoded data output device according to any one of claims27 to 38; decode means for decoding a base band signal from an input bitstream; and multiplex means for multiplexing encoded data, which isobtained by applying the encoded data output device to the input bitstream, to the base band signal.
 40. An encoded data restore device forrestoring the encoded data of respective macroblocks for encoding basedon the encoded data of respective macroblocks created by an encoded dataoutput device for outputting encoded data of respective macroblockscorresponding to an input bit stream in a predetermined number of bitsor less, the device comprising: prediction mode determination means fordetermining whether or not a prediction mode applied to the macroblocksis a predetermined prediction mode; encoded data selection determinationmeans for determining, in the encoded data output device, that encodeddata of a predetermined type corresponding to the pixels at thepredetermined positions in the macroblocks is selected in themacroblocks to which the predetermined prediction mode is applied; bitdistribution determination means for determining, in the encoded dataoutput device, that bits are more preferentially distributed to apredetermined type of encoded data selected at the encoded dataselection means than a predetermined type of encoded data that is notselected at the encoded data selection means in the macroblocks to whichthe predetermined prediction mode is applied; and encoded data restoremeans for restoring the encoded data of the respective macroblocks forrecoding based on a result of the bit distribution determination means.41. An encoded data restore device according to claim 40, wherein it isdetermined by the bit distribution determination means that thepredetermined type of encoded data, which is determined to be notselected in the encoded data output device by the encoded data selectiondetermination means, is reduced in the encoded data output device. 42.An encoded data restore device according to claim 40, wherein it isdetermined by the bit distribution determination means that bits, whichare the same or less than the bits in the input bit stream, aredistributed to or reduced from the predetermined type of encoded datawhich is determined to be selected in the encoded data output device bythe encoded data selection determination means in the encoded dataoutput device.
 43. An encoded data restore device according to claim 40,wherein the predetermined prediction mode is a mode for further dividingthe submacroblocks constituting each of the macroblocks to one or aplurality of secondary submacroblocks and arranging the secondarysubmacroblocks as motion compensation blocks.
 44. An encoded datarestore device according to claim 40, wherein the predeterminedpositions are four corners of the macroblocks.
 45. An encoded datarestore device according to claim 40, wherein when the predeterminedtype of coded data of other macroblock lacks, the predeterminedpositions are positions relating to the predetermined type of coded datautilized to reproduce the predetermined type of lacking coded data. 46.An encoded data restore device according to claim 40, wherein thepredetermined type of coded data includes at least a motion vector. 47.An encoded data restore device according to claim 40 wherein: predictingdirection determination means is applied to determine whether or not apredicting direction applied to the respective submacroblocks includedin the macroblocks, to which the predetermined prediction mode isapplied, is bidirection; and the encoded data selection means is appliedonly to the submacroblocks to which it is determined that the predictingdirection thereof is not bidirection at the predicting directiondetermination means.
 48. An encoded data restore device according toclaim 40, wherein: predicting direction determination means is appliedto determine whether or not a predicting direction applied to therespective submacroblocks included in the macroblocks, to which thepredetermined prediction mode is applied, is bidirection; and theencoded data selection means is applied to the submacroblocks to whichit is determined by the predicting direction determination means thatthe predicting direction thereof is not bidirection, and the encodeddata selection means is applied or not applied to the submacroblocks towhich it is determined by the predicting direction determination meansthat the predicting direction thereof is bidirection.
 49. An encodeddata restore device according to claim 40, wherein: predicting directiondetermination means is applied to determine whether or not a predictingdirection applied to the respective submacroblocks included in themacroblocks, to which the predetermined prediction mode is applied, isbidirection; and the encoded data selection means is applied to themacroblocks only when the number of the submacroblocks, which isincluded in the macroblocks to which the predetermined prediction modeis applied, and to which a bidirectional predicting direction isapplied, is a predetermined value or less.
 50. An encoded data restoredevice according to claim 40, wherein the encoded data selection meansdetermines that a predetermined type of encoded data, which correspondsto secondary submacroblocks including the pixels at the predeterminedpositions in the macroblocks, is selected in the macroblocks to whichthe predetermined prediction mode is applied, in the encoded data outputdevice.
 51. An encoded data restore device according to claim 40,wherein, as to a type of encoded data other than the predetermined type,the encoded data is restored based on that bits are not reduced in theencoded data output device.
 52. An encoding device comprising: therespective means of the encoded data restore device according to any oneof claims 40 to 51; and an encode means for encoding an input base bandsignal making use of the encoded data restored by the encoded datarestore device.
 53. A program for causing a computer to carry out themethod according to any one of claims 1 to 26.